Bremerton
Assessing Open-world Forgetting in Generative Image Model Customization
Laria, Héctor, Gomez-Villa, Alex, Marouf, Imad Eddine, Wang, Kai, Raducanu, Bogdan, van de Weijer, Joost
'"Close-up person in '"Street" all are smoking" Methods like Dreambooth lead to substantial drift in previously learned representations during the finetuning process even when adapting to as few as five images: a) Appearance drift: Columns demonstrate fine-grained class changes, complete object and scene shifts, and alterations in color (on both rows, images are generated from same seed). Recent advances in diffusion models have significantly enhanced image generation capabilities. However, customizing these models with new classes often leads to unintended consequences that compromise their reliability. We introduce the concept of open-world forgetting to emphasize the vast scope of these unintended alterations, contrasting it with the well-studied closed-world forgetting, which is measurable by evaluating performance on a limited set of classes or skills. Our research presents the first comprehensive investigation into open-world forgetting in diffusion models, focusing on semantic and appearance drift of representations. We utilize zero-shot classification to analyze semantic drift, revealing that even minor model adaptations lead to unpredictable shifts affecting areas far beyond newly introduced concepts, with dramatic drops in zero-shot classification of up to 60%. Additionally, we observe significant changes in texture and color of generated content when analyzing appearance drift. To address these issues, we propose a mitigation strategy based on functional regularization, designed to preserve original capabilities while accommodating new concepts. Our study aims to raise awareness of unintended changes due to model customization and advocates for the analysis of open-world forgetting in future research on model customization and finetuning methods. Furthermore, we provide insights for developing more robust adaptation methodologies. Recent advancements in image generation have led to the development of remarkably powerful foundational models capable of synthesizing highly realistic and diverse visual content. Techniques such as Generative Adversarial Networks (GANs) (Goodfellow et al., 2014), and more recently autoregressive models (Yu et al., 2022), Rectified Flows (Liu et al., 2023), and Denoising Diffusion Probabilistic Models (DDPMs) (Ho et al., 2020), have each contributed to significant progress in the field. These methods offer unique strengths in sample quality, diversity, and controllability. Among them, diffusion models have gained particular prominence due to their recent successes and growing influence, especially in enabling text-based image generation (Shonenkov et al., 2023; Ramesh et al., 2022) and complementary multimodal conditioning (Zhang & Agrawala, 2023; Mou et al., 2023), making them a key focus in current research and applications.
- North America > United States > Washington > Kitsap County > Bremerton (0.04)
- North America > Canada > Ontario > National Capital Region > Ottawa (0.04)
- Asia > China > Guangdong Province > Guangzhou (0.04)
- Leisure & Entertainment (0.93)
- Media (0.68)
MP2D: An Automated Topic Shift Dialogue Generation Framework Leveraging Knowledge Graphs
Hwang, Yerin, Kim, Yongil, Jang, Yunah, Bang, Jeesoo, Bae, Hyunkyung, Jung, Kyomin
Despite advancements in on-topic dialogue systems, effectively managing topic shifts within dialogues remains a persistent challenge, largely attributed to the limited availability of training datasets. To address this issue, we propose Multi-Passage to Dialogue (MP2D), a data generation framework that automatically creates conversational question-answering datasets with natural topic transitions. By leveraging the relationships between entities in a knowledge graph, MP2D maps the flow of topics within a dialogue, effectively mirroring the dynamics of human conversation. It retrieves relevant passages corresponding to the topics and transforms them into dialogues through the passage-to-dialogue method. Through quantitative and qualitative experiments, we demonstrate MP2D's efficacy in generating dialogue with natural topic shifts. Furthermore, this study introduces a novel benchmark for topic shift dialogues, TS-WikiDialog. Utilizing the dataset, we demonstrate that even Large Language Models (LLMs) struggle to handle topic shifts in dialogue effectively, and we showcase the performance improvements of models trained on datasets generated by MP2D across diverse topic shift dialogue tasks.
- Asia > South Korea > Seoul > Seoul (0.04)
- South America (0.04)
- North America > United States > Washington > Kitsap County > Bremerton (0.04)
- (2 more...)
- Leisure & Entertainment > Sports > Soccer (0.93)
- Media (0.93)
U.S. Aircraft Carrier Returning Home After Long Sea Tour Watching Iran
The aircraft carrier Nimitz is finally going home. The Pentagon last month ordered the warship to remain in the Middle East because of Iranian threats against President Donald J. Trump and other American officials, just three days after announcing the ship was returning home as a signal to de-escalate rising tensions with Tehran. With those immediate tensions seeming to ease a bit, and President Biden looking to renew discussions with Iran on the 2015 nuclear accord that Mr. Trump withdrew from, three Defense Department officials said on Monday that the Nimitz and its 5,000-member crew were ordered on Sunday to return to the ship's home port of Bremerton, Wash., after a longer-than-usual 10-month deployment. The Pentagon for weeks had been engaged in a muscle-flexing strategy aimed at deterring Iran and its Shia proxies in Iraq from attacking American personnel in the Persian Gulf to avenge the death of Maj. General Suleimani, the commander of Iran's elite Quds Force of the Islamic Revolutionary Guards Corps, was killed in an American drone strike in January 2020.
- Indian Ocean > Arabian Gulf (0.68)
- Asia > Middle East > Saudi Arabia > Arabian Gulf (0.68)
- North America > United States > Washington > Kitsap County > Bremerton (0.28)
- (4 more...)
- Government > Regional Government > North America Government > United States Government (1.00)
- Government > Military (1.00)
- Government > Regional Government > Asia Government > Middle East Government > Iran Government (0.62)
In Abrupt Reversal of Iran Strategy, Pentagon Orders Aircraft Carrier Home
The Pentagon has abruptly sent the aircraft carrier Nimitz home from the Middle East and Africa over the objections of top military advisers, marking a reversal of a weekslong muscle-flexing strategy aimed at deterring Iran from attacking American troops and diplomats in the Persian Gulf. Officials said on Friday that the acting defense secretary, Christopher C. Miller, had ordered the redeployment of the ship in part as a "de-escalatory" signal to Tehran to avoid stumbling into a crisis in President Trump's waning days in office. American intelligence reports indicate that Iran and its proxies may be preparing a strike as early as this weekend to avenge the death of Maj. Senior Pentagon officials said that Mr. Miller assessed that dispatching the Nimitz now, before the first anniversary this Sunday of General Suleimani's death in an American drone strike in Iraq, could remove what Iranian hard-liners see as a provocation that justifies their threats against American military targets. Some analysts said the return of the Nimitz to its home port of Bremerton, Wash., was a welcome reduction in tensions between the two countries.
- Europe > Middle East (0.42)
- Africa > Middle East (0.42)
- North America > United States > Washington > Kitsap County > Bremerton (0.28)
- (4 more...)
- Government > Regional Government > North America Government > United States Government (1.00)
- Government > Military (1.00)